/*
=============================================================
File: OptimizationFunctions.txt    
Link: https://oeis.org/wiki/File:OptimizationFunctions.txt
=============================================================
*/


/* Notes:
=============================================================
25 Oct 2014:

The MinSearch algorithm is based on the Brent method, and it
is basically copied from Recipes. However, I had to modify
the termination criterium because with the arbitrary precision
arithmetic of PARI, it was very often hanging in never-ending
loops. The ad-hoc modification seems to work in the sense that
a fast and reasonable termination seems to occur always.
Nevertheless, the function is usable but not satisfactory,
because it does not provide any indication of the final
precision. Usually, it is limited by the fact that, up to the
current default(realprecision), the value of the maximum does
not vary over an interval of x values much wider than what
default(realprecision) would indicate. This forces the User
to run the task with much higher and increasing precision(s)
and laboriously compare the individual results. 

Clearly, more work is called for and, in the end, this version
is likely to become deprecated.

As a challenging test, I have used a search of the first (and
only) maximum in the Dirichlet eta function eta(z) for z=x*I
and real x. To do that, f(x) was defined as
gp> f(x)=-eta(x*I,1)
and the search call looked like
gp> xmax=MinSearch(0.01,1.0,10.0,f)
To achieve 2000 digits precision in the xmax value, the needed
default precision was found to be around 16000 digits (20000
were used for the actual published data).
The results are in A097078 and A097079.

=============================================================
*/

MinSearch(ax,bx,cx,f,monit=0,itermax=10000) =
/* ----------------------------------------------------------
Searches for minimum of a real function f(x) in the interval
[ax,cx]. bx is an externally supplied raw estimate of the
minimum location. It is required that f(ax)>f(bx) and
f(cx)>f(bx). Returns the location of the minimum.
Attention: The error in the minimum location can be much
larger than the current precision. It depends more or less
on the value of second derivative of f at xmin. When the
derivative values are of the order of units, for example,
the correct digits of xmin can be much less than
default(realprecision). It is essential to run the search
several times with various precisions and compare the results.
---------------------------------------------------------- */
{
  my(a,b,d,etemp,x,xm,v,w,u,fx,fv,fw,fu,p,q,r,e=0.0,
     rtol=10.0^(-default(realprecision)\2),atol,iter=1);
  if(ax<cx, a=ax;b=cx, a=cx;b=ax); \\ Initializations
  x=bx;v=bx;w=bx;
  fx=f(x);fv=fx;fw=fx;
  while (iter<=itermax,iter++; \\ Main loop
    if(monit,print(x));
    xm=(a+b)/2;
    atol = rtol*rtol*(abs(a)+abs(b));
    if (abs(x-xm)<=atol,return(xm)); \\ Test for done
    if (abs(e) > rtol,
      r = (x-w)*(fx-fv); \\ Construct a trial parabolic fit
      q = (x-v)*(fx-fw);
      p = (x-v)*q-(x-w)*r;
      q = 2*(q-r);
      if (q>0, p=-p);
      q = abs(q);
      etemp = e;
      e = d;
      if ((abs(p)>=abs(q*etemp/2))||(p<=q*(a-x))||(p>=q*(b-x)),
        if(x>=xm,e=a-x,e=b-x); \\ Parabolic step unacceptable
        d=0.381966*e; \\ Do approx.golden-ratio cut
        ,
        d = p/q; \\ Take parabolic step
        u = x+d;
        if (((u-a)<atol)||((b-u)<atol),d=atol;if(x>xm,d=-d)); 
      );  
      , 
      if (x>=xm,e=a-x,e=b-x); \\ Do approx.golden-ratio cut
      d=0.381966*e;
    );
    if (abs(d)>=atol, u=x+d, u=x+atol*sign(d));
    fu = f(u); \\ Function evaluation
    if (fu<=fx,
      if (u>=x, a=x, b=x);
      v=w;w=x;x=u;
      fv=fw;fw=fx;fx=fu;  
      ,
      if (u<x, a=u, b=u);
      if ((fu<=fw)||(w==x),
        v=w;fv=fw;w=u;fw=fu;
        ,
        if ((fu<=fv)||(v==x)||(v==w), v=u;fv=fu;);
      );
    ); 
  );
  print("Exceeded ",itermax," iterations");
  return(x);
}


/*
=============================================================
  Contributed to OEIS Wiki by Stanislav Sykora
=============================================================
*/

